Model performance through text-to-text transformation via distant supervision from target and auxiliary tasks

ABSTRACT

A computer-implemented method of performing text-to-text transformation includes performing a text transformation operation on an original input text of a specific task to generate a plurality of transformed text. A task-specific performance metric that measures an operation of the specific task is applied to each one of the plurality of transformed text. Each of the plurality of transformed text are paired with the task-specific performance metric. A training dataset is updated to include each pairing of the plurality of transformed text with the task-specific metric.

BACKGROUND Technical Field

The present disclosure generally relates to systems and methods ofNatural Language Processing, and more particularly, to text-to-texttransformation performed by statistical models.

Description of the Related Art

Natural Language Processing (NLP) is used to teach computers tounderstand how humans interact with machines both via input speech andtext. Some NLP problems can be solved more accurately and efficientlywith a transformed version of an original input. Text-to-texttransformation is a way to modify the original input from a user to anew form that works better for the computer to perform the specifictask(s) of interest to a user.

There are shortcomings in the application of text-to-text transformationto provide for more accurate and efficient NLP, including but notlimited to performing a target task. In general, text-to-texttransformation is performed using statistical models trained on labeleddata. However, training example for such transformation operations arenot always available, and the generation of labeled data can be timeconsuming and expensive to perform a particular target task.

SUMMARY

According to one embodiment, a computer-implemented method of performingtext-to-text transformation includes performing a text transformationoperation on an original input text of a specific task to generate aplurality of transformed text. A task-specific performance metric thatmeasures an operation of the specific task is applied to each one of theplurality of transformed text. Each of the plurality of transformed textare paired with the task-specific performance metric. A training datasetis updated to include each pairing of the plurality of transformed textwith the task-specific metric. This text-to-text transformationincreases the performance of a model by providing better data fortraining and obtaining inferences.

In one embodiment, each pairing is provided to the training datasetafter determining whether a quality criterion is satisfied. Thelower-quality pairings are not providing to the training dataset toincrease the accuracy and speed of a model.

In one embodiment, the specific task includes at least one target taskprovided in the original text. The method further includes: performing adistant supervision of the text transformation operation based on a setof related auxiliary tasks having labeled data in addition to a data ofthe at least one target task. The distant supervision allows for moreefficient generating of training data by using the labeled data ofauxiliary tasks as part of the training dataset. A more accurate modelis trained in cases where there is little or no training data for theoriginal input text, and the time and costs of generating a trainingdataset for the model is reduced.

In one embodiment, the computer-implemented method further includesperforming a new text transformation operation using the updatedtraining dataset to generate a new plurality of transformed text. Thetask-specific performance metric is applied to measure an operation ofthe specific task using each of the new plurality of transformed text.Each of the new plurality of transformed text is paired with thetask-specific performance metric, and the training dataset is updated toinclude each pairing of the new plurality of transformed text with thetask-specific metric. A more accurate and faster answer to an input canbe provided by the model by using the transformed text, rather than theoriginal input text, to perform a task.

In one embodiment, the labeled data of the related auxiliary tasksincludes question and answer (QA) pairs. The text transformationoperation further includes generating a plurality of questioncompression candidates from the QA pairs and selecting one or more ofthe compression candidates based on an answer ranking of the QA pairs.Question compression and QA pairing provides for accurate training andspeed in the performance of a model.

In one embodiment, the question compression candidates are based on anumber of words in the original text. The text transformation operationincludes compressing questions by providing a summary of the questioncompression candidates using fewer words than in the original text.Question compression provides for accurate training and speed in theperformance of a model.

In one embodiment, the updating of the training data set is performediteratively after performing a successively new text transformationoperation, applying the task-specific performance metric, and pairingeach successively new transformed text until a terminating criterion issatisfied. The iterations enhance the accuracy of the training dataset.

In one embodiment, for each successively new text transformationoperation, the pairing of the newly successive transformed text and thetask-specific performance metric is provided to the training datasetafter determining whether a quality criterion is satisfied. The use ofthe quality criterion enhances the accuracy of the training dataset.

In one embodiment, the specific task is answering a question.

In one embodiment, the text transformation operation further includesproviding an answer to a question embedded in one or more one orrelevant portions of the question. More accurate answers to questionscan be provided is less time by using the relevant portions of a model.

In one embodiment, the providing of an answer includes discarding atleast one non-selected portion of the question upon determining that thenon-selected portion of the question is at least one of irrelevant toproviding the answer to the question, or redundant to at least a part ofthe selected relevant portions of the question. More accurate answers toquestions can be provided is less time by using the relevant portions ofa model.

According to one embodiment, a computer-implemented method of performingtext-to-text transformation includes receiving training examples relatedto an original text of a specific task. A text-to-text transformationoperation of an original text is performed to generate training examplesof a transformed text. A machine learning model is trained to performNatural Language Processing (NLP) based upon a training data comprisingthe training examples of the original text and the training examples ofthe transformed text. A test data including a new original text fortransformation is received. The new original text is transformed intonew transformed text using the trained machine learning model. Thistext-to-text transformation increases the performance of a model byproviding better training data.

In one embodiment, the generated training examples of the transformedtext includes a summarizing of the original text. The summary provides amore succinct form of the input text, and the model can perform a taskrelated to the input text more accurately using the summary.

In one embodiment, the generated training examples of the transformedtext includes a summarizing and a shortening of the original text. Themodel can provide a more accurate response by summarizing and shorteningthe original text.

In one embodiment, the specific task includes at least one target taskprovided in the original text. The method further includes performing adistant supervision of the text transformation operation based on a setof related auxiliary tasks having labeled data in addition to a data ofthe at least one target task. The distant supervision allows for moreefficient generating of training data by using the labeled data ofauxiliary tasks as part of the training dataset. A more accurate modelis trained in cases where there is little or no training data for theoriginal input text, and the time and costs of generating a trainingdataset for the model is reduced.

According to an embodiment, a computing device configured to performtext-to-text transformation includes a processor, a memory coupled tothe processor, the memory storing instructions to cause the processor toperform acts that include performing a text transformation operation onan original input text of a specific task to generate a plurality oftransformed text. A task-specific performance metric is applied tomeasure an operation of the specific task using each one of theplurality of transformed texts. Each of the plurality of the transformedtext is paired with the task-specific performance metric. A trainingdataset is updated to include each pairing of the plurality oftransformed text with the task-specific metric. The text-to-texttransformation increases the performance of a model by providing bettertraining data.

In one embodiment, the specific task includes at least one target taskprovided in the original text. The instructions cause the processor toperform an additional act of performing a distant supervision of thetext transformation operation based on a set of related auxiliary taskshaving labeled data in addition to a data of the at least one targettask. The distant supervision allows for more efficient generating oftraining data.

In one embodiment, the labeled data of the related auxiliary tasksincludes question and answer (QA) pairs. The instructions cause theprocessor to perform additional acts of generating a plurality ofquestion compression candidates from the QA pairs and selecting one ormore of the compression candidates based on an answer ranking of the QApairs. Question compression and QA pairing provides for accuratetraining and speed in the performance of a model.

In one embodiment, the question compression candidates are based on anumber of words in the original text. The instructions cause theprocessor to perform an additional act of compressing questions in thetext transformation operation by providing a summary of the questioncompression candidates using fewer words than in the original text.Question compression provides for accurate training and speed in theperformance of a model.

In one embodiment, the instructions cause the processor to performadditional acts of performing a new text transformation operation usingthe updated training dataset to generate a new plurality of transformedtext. The task-specific performance metric is applied to measure anoperation of the specific task using each one of the new plurality oftransformed text. Each of the new plurality of transformed text ispaired with the task-specific performance metric. The training datasetis updated to include each pairing of the new plurality of transformedtext with the task-specific metric. The additional transformationsenhance the accuracy of the training dataset.

These and other features will become apparent from the followingdetailed description of illustrative embodiments thereof, which is to beread in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings are of illustrative embodiments. They do not illustrate allembodiments. Other embodiments may be used in addition to or instead.Details that may be apparent or unnecessary may be omitted to save spaceor for more effective illustration. Some embodiments may be practicedwith additional components or steps and/or without all the components orsteps that are illustrated. When the same numeral appears in differentdrawings, it refers to the same or like components or steps.

FIG. 1 is an overview of a text-to-text transformation operation,consistent with an illustrative embodiment.

FIG. 2 illustrates some non-exhaustive examples of question compression,consistent with an illustrative embodiment.

FIG. 3 illustrates an operation of training a model for performingtext-to-text transformation, consistent with an illustrative embodiment.

FIG. 4 illustrates answer ranking operations using selected bestcompressions, consistent with an illustrative embodiment.

FIG. 5 illustrates a decoding operation in which long questions arereduced in length, consistent with an illustrative embodiment.

FIG. 6 illustrates the operation of a natural language processing modelperforming a duplicate question detection task utilizing text-to-texttransformation, consistent with an illustrative embodiment.

FIG. 7 illustrates the difference in performance of another naturallearning process model performing a duplicate question detection taskwith ranked results, consistent with an illustrative embodiment.

FIG. 8 is a flowchart illustrating a computer-implemented method ofperforming text-to-text transformation, consistent with an illustrativeembodiment.

FIG. 9 is a functional block diagram illustration of a computer hardwareplatform, consistent with an illustrative embodiment.

FIG. 10 depicts an illustrative cloud computing environment, consistentwith an illustrative embodiment.

FIG. 11 depicts a set of functional abstraction layers provided by acloud computing environment, consistent with an illustrative embodiment.

DETAILED DESCRIPTION Overview

In the following detailed description, numerous specific details are setforth by way of examples to provide a thorough understanding of therelevant teachings. However, it should be understood that the presentteachings may be practiced without such details. In other instances,well-known methods, procedures, components, and/or circuitry have beendescribed at a relatively high level, without detail, to avoidunnecessarily obscuring aspects of the present teachings.

As used herein, the term “weak supervision” relates to a type ofsupervised learning, with uncertainty in the labeling based on, forexample, automatic labeling or because the data was labeled bynon-experts.

The term “distant supervision” as used herein is to be understood as aform of weak supervision in which an auxiliary automatic mechanism isused to produce weak labels/reference output (e.g., without the databeing labeled by a non-expert).

In the present disclosure, a T3 (e.g., a text-to-text transformation)framework is used to improve performance on the target task. However,the computer-implemented method and device according to the presentdisclosure is not limited to using any particular framework.

In an illustrative embodiment, a text-to-text-transformation operationis decoupled from the target task to enhance performance on the targettask. By training a model to use the target task to provide distantsupervision to the text-to-transformation operation, there is no needfor additional annotation data to perform the text-to-texttransformation. In addition, the use of auxiliary tasks that are similarbut separate from the target task for which labeled data is alreadyavailable to train model to perform the text-to-text transformationoperation.

In an overview, when NLP is used to receive questions that arerelatively long, it is beneficial for NLP systems to understand andextract key points before providing an answer. In cases where theoriginal input is a poorly-worded question, it is also beneficial forNLP to perform query reformulation and/or expansion of the originalinput to improve the performance of an NLP model in terms of accuracyand speed.

FIG. 1 is an overview 100 illustrating a text-to-text transformationoperation, consistent with an illustrative embodiment. FIG. 1 showsthree operations labeled 1, 2 and 3, respectively. The first operationtrains examples for text-to-text transformation (T3) 120 by utilizingdistant supervision for candidate generation 115 of the original targettask 110 and an auxiliary task, and training examples from the targetand auxiliary tasks 115.

The second operation shows that training examples 135 are used to traina text-to-transformation model 140 to provide transformed text 145(referred to as T3). In the third operation target task and trainingtext examples 165 utilizing the transformed text T3 is provided to ainput 170 (referred to as a transformed input 170 of the original data)to train and/or decode a target task model 175.

The computer-implemented method and computer device of the presentdisclosure advantageously provides improved performance of input textsuch as questions by transforming the text into a form that is shorterand often filters extraneous or irrelevant information. An NLP model canfind answers more quickly and accurately and constitutes an improvementin text-to-text transformation, as well as an improvement in computeroperations. Through the use of distant supervision, the NLP model can betrained with labeled data from auxiliary tasks that are related to aspecific task. By virtue of the teachings herein, a reduction inprocessing overhead and storage can be realized, as well as a reductionin power consumed.

Additional advantages of the computer-implemented method and device ofthe present disclosure are disclosed herein.

Example Embodiment

FIG. 2 illustrates some non-exhaustive examples 200 of questioncompression, consistent with an illustrative embodiment. In FIG. 2, thecompressed questions 210 provides for a more accurate model performancethan the original question 205 because information that is determined tobe irrelevant or not one of the main points of the question has beenremoved. For example, the portion 215 of the original question 205 thatasks about what teenagers do in Doha can cause an NLP model to provideresults that are not entirely on point in response to the question. Forexample, in response to portion 215, the NLP model may search foractivities for 13 and 14 year old's (teenagers) even though another partof the question specifically states 17-18 year old's. While a human maybetter understand a question by including context, an NLP model may havea slower response and provide less accurate responses because it lacksthe nuance of human thought. The boldface portions of the questions suchas asking about what are the happenings in Qatar on holidays 225 is morenarrow than the more general “what to do on EID holidays?” which may beunderstood by an NLP system as being anywhere in the world. Thus,responses that do not have anything to do with Qatar may be provided.Also, the portion of the question of the happenings in Qatar on holidays225 did not make reference to the EID holiday, so results for otherholiday happenings in Qatar may be provided.

FIG. 3 illustrates an operation 300 of training a model for performingtext-to-text transformation, consistent with an illustrative embodiment.The transforming of questions for answer ranking is used for evaluatingthe Question-Answer (QA) pairs that have been generated by thetext-to-text transformation process (transforming original text intotransformed text that may be shorter in length or more focused than theoriginal text). As previously discussed, auxiliary tasks that arerelated to a target task can be used as training examples 305 to generallabeled data by distant supervision. QA pairs 305 are evaluated by aranker training 310 that is trained to rank the QA pairs according tocertain criteria, such as compression or accuracy. Such used by theSummarizer Training algorithm 330 to produce a Summarizer Model 335.

QA Pairs 305 are used by the Train Ranker algorithm 310 to train RankerModel 315. Each QA Pair 305 is also used by the generate questioncompression candidates algorithm 312 to produce a plurality ofcompression candidates 314. Every compression candidate 314corresponding to the same training example for ranking is evaluatedusing ranker model 315 by the Select Best Compression based on RankingPerformance algorithm 320, which uses each of a plurality of thecompression candidates 314 corresponding to each QA Pair 305 todetermine its ranking performance. Step 320 selects the best compressionfrom the plurality of compression candidates 314 corresponding to eachthe training example for ranking. The collection of selected bestcompression candidates corresponding to all Training Examples forRanking 304 constitutes the Best Compressions 325 which are used by theSummarizer Training algorithm 330 to produce a Summarizer Model 335.

The ranker model 315 is configured to receive the ranked QA pairs fromthe train ranker 310 and run tests on the QA pairs to determine theranking performance. The generate question compression candidates module312 is configured to generate compression candidates 314 that, forexample, may be reduced in size or scope to improve the accuracy of theperformance of a specific task. There is a selection of a bestcompression operation 320 based of the compression candidates 314 andthe ranking performance of the QA pairs by the ranker model 315. Thebest compressions 325, which may be the compression with the highestperformance (e.g., as evaluated by a specific performance task metric)are forwarded to a summarizer trainer 330 module. The summarizer trainer330 then trains the summarizer model 335 to transform text bysummarizing of the original input text or additional text as may beprovided into a transformed text.

FIG. 4 illustrates answer ranking operations using respectivelydifferent forms of selected best compressions, consistent with anillustrative embodiment. The answer ranking operations may be performedfor duplicate tasks. A long question is compressed into a shorterversion for processing by the model. The system is trained to createshorter questions. Shorter questions are often more direct and the modelmay process a response to an original text input in less time, usingfewer computer resources.

With reference to FIG. 4, a long question 445 (an original input text)is decoded 440 with a resultant compressed question. The decoding mayinclude identifying portions of the question (see FIG. 2 for theboldface type). The portions may include several words, phrases or asentence of a multiple sentence question. The compressed question can beconsidered to be transformed text as compared with the long question445. An evaluation metric 430 is a task-specific performance metric thatoperates to evaluate the performance of the specific task identified inthe compressed question 435. The compressed question 435 may include,for example, a plurality of compressed sentences or compressed phrases.Each portion of the transformed text of the compressed question ispaired with the task-specific performance metric. A Ranker 405 istrained to rank the portions of the compressed question (such ascompressed sentence or compressed phrases). The ranker 405 may betrained to rank the compressed questions with labeled data from one ormore auxiliary tasks from similar tasks. There is a decoding operation410 to identify the ranked sentence compression in order. At operation415 a, the best compression (e.g., one or more compressions with thehighest ranking) is selected, and the ranker is trained 420 to rank thebest compressions. In addition, a summarizer 425 is trained with theselected best compressions. The summarizer, by being trained with theselected best compression, is able to summarize the long questions moreaccurately.

FIG. 5 illustrates a decoding operation in which long questions arereduced in length, consistent with an illustrative embodiment. The longquestions 501 are summarized by the summarizer model 505. The summarizertransforms the long questions into compressed questions 515 and isprovided to the QA model 525, which provides answers to the questions520. The QA model may have labeled data from auxiliary tasks that are inthe form of QA pairs from similar questions related to auxiliary tasksof the specific task. In other words, there is some commonality betweenthe auxiliary tasks and the specific task, but they are not identical.

FIG. 6 illustrates the performance of an NLP model performing aduplicate question detection task utilizing text-to-text transformation,consistent with an illustrative embodiment. The task performed wasduplication question detection. In this embodiment, an input text (e.g.,a question) is transformed by performing questioncompression/summarization by dropping sentences (in this particularembodiment). Then, the evaluation metric utilizing mean averageprecision (MAP) provides an accuracy rating of the original data and thecompression data. In table 605 it can be seen that thetext-to-transformation model 615 has an improved accuracy when comparedwith the computing device performing the specific task with the originalinput text 610.

FIG. 7 illustrates the difference in performance 700 of another NLPmodel performing a duplicate question detection task with rankedresults, consistent with an illustrative embodiment. The table 705 showsthe use of a baseline ranker 710 versus a compressed ranker 715 in theevaluation process of a device dataset and a test data set. The originaldata shows slightly better accuracy in performance using the baselineranker 71. However, selecting the best compression shows a much higheraccuracy in performance of a specific task, and the compressed ranker715 shows a slightly higher accuracy than the baseline ranker 715. Thethird comparison is with transformation of the original data aftertransformation, where there is not a selection of the best compression.It is shown that using data as ranked by the compressed ranker providesmuch better results than the baseline ranker 715 and close to the bestcompression accuracy, but definitely less than the accuracy using thebest compressions.

Example Process

With the foregoing overview of the example architecture, it may behelpful now to consider a high-level discussion of an example process.To that end, FIG. 8 is a flowchart illustrating a computer-implementedmethod of performing text-to-text transformation, consistent with anillustrative embodiment. FIG. 8 is shown as a collection of blocks, in alogical order, which represents a sequence of operations that can beimplemented in hardware, software, or a combination thereof. In thecontext of software, the blocks represent computer-executableinstructions that, when executed by one or more processors, perform therecited operations. Generally, computer-executable instructions mayinclude routines, programs, objects, components, data structures, andthe like that perform functions or implement abstract data types. Ineach process, the order in which the operations are described is notintended to be construed as a limitation, and any number of thedescribed blocks can be combined in any order and/or performed inparallel to implement the process.

At operation 805, a text transformation operation is performed on anoriginal input text of a specific task to generate a plurality oftransformed text. For example, an input question 205 such as shown inFIG. 2 is transformed into a compressed question 210. As previouslydiscussed, the text transformation operation is in some illustrativeembodiments an iterative process that may be performed until certainterminating criterion has been reached. The criterion may includequality as evaluated by an evaluating algorithm with mean averageprecision (MAP), or some other way to measure the operation of thespecific task with the original data versus the transformed data.

At operation 815, a task-specific performance metric is applied tomeasure an operation of the specific task using each one of theplurality of transformed text. For example, after a duplicate questiondetection operation is performed on an input text, a ranked score oftask-specific performance metric can be used on the various iterationsof transformed text and compared with the original input data.

At operation 820, a pairing each of the plurality of transformed textwith the task-specific performance metric is performed. For example,Question-Answer (QA) pairs are generated of portions of the transformedtext and a task-specific performance metric.

At operation 825, there is performed an updating of a training datasetto include each pairing of the plurality of transformed text with thetask-specific performance metric. For example, the QA generated atoperation 820 are collected and added to the training data. There can bea selective updating of training data to increase the accuracy of themodel to perform the specific task. QA pairs that are ranked lower thana certain metric, or ranked relatively lower than other QA pairs may beexcluded from the updating of the training data. Thus, training data canbe provided using distant supervision by labelling data without using anexpert or having to access an external knowledge base, or by usingauxiliary data such as emojis, hashtags, or URLs that exist in socialmedia or other similar resource.

Example Particularly Configured Computer Hardware Platform

FIG. 9 provides a functional block diagram illustration 900 of acomputer hardware platform. In particular, FIG. 9 illustrates aparticularly configured network or host computer platform 900, as may beused to implement the method shown in FIG. 8.

The computer platform 900 may include a central processing unit (CPU)904, a hard disk drive (HDD) 906, random access memory (RAM) and/orread-only memory (ROM) 908, a keyboard 910, a mouse 912, a display 914,and a communication interface 916, which are connected to a system bus902. The HDD 906 can include data stores.

In one embodiment, the HDD 906 has capabilities that include storing aprogram that can execute various processes, such as machine learning,text-to-text transformation, and question compression.

In one embodiment, the HDD 906 has capabilities that include storing aprogram that can execute various processes, such as machine learning,text-to-text transformation, and question compression.

In FIG. 9, there are various modules shown as discrete components forease of explanation. However, it is to be understood that thefunctionality of such modules and the quantity of the modules may befewer or greater than shown.

The text-to-text-transformation module 940 is configured to control theoperation of the modules 942-952 to perform the operations oftext-to-text transformation, consistent with an illustrative embodiment.One such application is to modify the input from a user to a new formthat works better for a specific task of interest to the user byreducing results based on irrelevant or redundant parts of a longquestion. For example, the compression module 942 is configured toreduce a question length of input text such as shown in FIG. 2. Thequestions may be reduced by several words or phrases, or even severalsentences in length. The reduction of words, phrases and/or sentences inthe questions provided is performed in conjunction with the distantsupervision module 944. The distant supervision module 944 is configuredto perform distant supervision of the training and operation of a modelto perform text-to-text transformation by providing labeled data fromauxiliary tasks to train the model to perform a target task. Theauxiliary tasks are similar to the target task in terms of, for example,a subject or category, but may not have some of the details of thetarget task based on an original input text. The labeled data of theauxiliary tasks assists in the generation of labeled data for the targettask, as the target task may have little or no labeled data.

The ranking model 946 is configured to perform an evaluation of varioustest-to-text transformations that occur in an iterative process that mayoccur in terms of question and answer ranking. The ranking module 946may also rank various QA pairs that are created, and the QA pair orpairs with the highest ranking scores may be used to perform the targettask. As shown in FIGS. 6 and 7, evaluating accuracy scores with meanaverage precision (MAP) are shown. The decoding module 948 is configuredto assist in training and operation of a Summarizer Model 505 and a QAModel 525 as shown in FIG. 5. Long questions can be summarized so as tobe shorter and more focused in content, and more accurate QA pairs canbe generated to perform a specific task.

The reinforcement learning module 950 is configured to train NLP modelsin text-to-text transformation, and can be used in distant supervisionfor automated label generation. The communication interface module 954is configured to receive original test input and to transmit thegenerated QA pairs and responses to target task to users of NLP system.

Example Cloud Platform

As discussed above, functions relating to the low bandwidth transmissionof high definition video data may include a cloud. It is to beunderstood that although this disclosure includes a detailed descriptionof cloud computing as discussed herein below, implementation of theteachings recited herein is not limited to a cloud computingenvironment. Rather, embodiments of the present disclosure are capableof being implemented in conjunction with any other type of computingenvironment now known or later developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g., networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as Follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported, providing transparency for both theprovider and consumer of the utilized service.

Service Models are as Follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as Follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service-oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure that includes anetwork of interconnected nodes.

Referring now to FIG. 10, an illustrative cloud computing environment1000 utilizing cloud computing is depicted. As shown, cloud computingenvironment 1000 includes cloud 1050 having one or more cloud computingnodes 1010 with which local computing devices used by cloud consumers,such as, for example, personal digital assistant (PDA) or cellulartelephone 1054A, desktop computer 1054B, laptop computer 1054C, and/orautomobile computer system 1054N may communicate. Nodes 1010 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 1000 to offerinfrastructure, platforms, and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 1054A-Nshown in FIG. 10 are intended to be illustrative only and that computingnodes 1010 and cloud computing environment 1000 can communicate with anytype of computerized device over any type of network and/or networkaddressable connection (e.g., using a web browser).

Referring now to FIG. 11, a set of functional abstraction layers 1100provided by cloud computing environment 1000 (FIG. 10) is shown. Itshould be understood in advance that the components, layers, andfunctions shown in FIG. 11 are intended to be illustrative only andembodiments of the disclosure are not limited thereto. As depicted, thefollowing layers and corresponding functions are provided:

Hardware and software layer 1160 include hardware and softwarecomponents. Examples of hardware components include: mainframes 1161;RISC (Reduced Instruction Set Computer) architecture-based servers 1162;servers 1163; blade servers 1164; storage devices 1165; and networks andnetworking components 1166. In some embodiments, software componentsinclude network application server software 1167 and database software1168.

Virtualization layer 1170 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers1171; virtual storage 1172; virtual networks 1173, including virtualprivate networks; virtual applications and operating systems 1174; andvirtual clients 1175.

In one example, management layer 1180 may provide the functionsdescribed below. Resource provisioning 1181 provides dynamic procurementof computing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 1182provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may include applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 1183 provides access to the cloud computing environment forconsumers and system administrators. Service level management 1184provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 1185 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 1190 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 1191; software development and lifecycle management 1192;virtual classroom education delivery 1193; data analytics processing1194; transaction processing 1195; and a text-to-text transformationmodule 1196 configured to perform text-to-text transformation throughdistant supervision, as discussed herein above.

CONCLUSION

The descriptions of the various embodiments of the present teachingshave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

While the foregoing has described what are considered to be the beststate and/or other examples, it is understood that various modificationsmay be made therein and that the subject matter disclosed herein may beimplemented in various forms and examples, and that the teachings may beapplied in numerous applications, only some of which have been describedherein. It is intended by the following claims to claim any and allapplications, modifications, and variations that fall within the truescope of the present teachings.

The components, steps, features, objects, benefits, and advantages thathave been discussed herein are merely illustrative. None of them, northe discussions relating to them, are intended to limit the scope ofprotection. While various advantages have been discussed herein, it willbe understood that not all embodiments necessarily include alladvantages. Unless otherwise stated, all measurements, values, ratings,positions, magnitudes, sizes, and other specifications that are setforth in this specification, including in the claims that follow, areapproximate, not exact. They are intended to have a reasonable rangethat is consistent with the functions to which they relate and with whatis customary in the art to which they pertain.

Numerous other embodiments are also contemplated. These includeembodiments that have fewer, additional, and/or different components,steps, features, objects, benefits and advantages. These also includeembodiments in which the components and/or steps are arranged and/orordered differently.

The flowchart, and diagrams in the figures herein illustrate thearchitecture, functionality, and operation of possible implementationsaccording to various embodiments of the present disclosure.

While the foregoing has been described in conjunction with exemplaryembodiments, it is understood that the term “exemplary” is merely meantas an example, rather than the best or optimal. Except as statedimmediately above, nothing that has been stated or illustrated isintended or should be interpreted to cause a dedication of anycomponent, step, feature, object, benefit, advantage, or equivalent tothe public, regardless of whether it is or is not recited in the claims.

It will be understood that the terms and expressions used herein havethe ordinary meaning as is accorded to such terms and expressions withrespect to their corresponding respective areas of inquiry and studyexcept where specific meanings have otherwise been set forth herein.Relational terms such as first and second and the like may be usedsolely to distinguish one entity or action from another withoutnecessarily requiring or implying any such actual relationship or orderbetween such entities or actions. The terms “comprises,” “comprising,”or any other variation thereof, are intended to cover a non-exclusiveinclusion, such that a process, method, article, or apparatus thatcomprises a list of elements does not include only those elements butmay include other elements not expressly listed or inherent to suchprocess, method, article, or apparatus. An element proceeded by “a” or“an” does not, without further constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises the element.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments have more featuresthan are expressly recited in each claim. Rather, as the followingclaims reflect, the inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus, the following claimsare hereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

What is claimed is:
 1. A computer-implemented method of performingtext-to-text transformation, the method comprising: performing a texttransformation operation on an original input text of a specific task togenerate a plurality of transformed text; applying a task-specificperformance metric to measure an operation of the specific task usingeach one of the plurality of transformed text; pairing each of theplurality of transformed text with the task-specific performance metric;and updating a training dataset to include each pairing of the pluralityof transformed text with the task-specific performance metric.
 2. Thecomputer-implemented method of claim 1, wherein each pairing is providedto the training dataset after determining whether a quality criterion issatisfied.
 3. The computer-implemented method of claim 1, wherein thespecific task comprises at least one target task provided in theoriginal text; and the method further comprising performing a distantsupervision of the text transformation operation based on a set ofrelated auxiliary tasks having labeled data in addition to a data of theat least one target task.
 4. The computer-implemented method of claim 3,further comprising: performing a new text transformation operation usingthe updated training dataset to generate a new plurality of transformedtext; applying the task-specific performance metric to measure anoperation of the specific task using each one of the new plurality oftransformed text; pairing each of the new plurality of transformed textwith the task-specific performance metric; and updating the trainingdataset to include each pairing of the new plurality of transformed textwith the task-specific metric.
 5. The computer-implemented method ofclaim 3, wherein: the labeled data of the related auxiliary tasksincludes question and answer (QA) pairs; and the text transformationoperation further comprises: generating a plurality of questioncompression candidates from the QA pairs; and selecting one or more ofthe compression candidates based on an answer ranking of the QA pairs.6. The computer-implemented method of claim 5, wherein: the questioncompression candidates are based on a number of words in the originaltext; and the text transformation operation includes compressingquestions by providing a summary of the question compression candidatesusing fewer words than in the original text.
 7. The computer-implementedmethod of claim 1, wherein the updating of the training data set isperformed iteratively after performing a successively new texttransformation operation, applying the task-specific performance metric,and pairing each successively new transformed text until a terminatingcriterion is satisfied.
 8. The computer-implemented method of claim 7,wherein for each successively new text transformation operation, thepairing of the newly successive transformed text and the task-specificperformance metric is provided to the training dataset after determiningwhether a quality criterion is satisfied.
 9. The computer-implementedmethod of claim 1, wherein the specific task comprises answering aquestion embedded in the input text.
 10. The computer-implemented methodof claim 9, wherein the text transformation operation further comprisesproviding an answer to a question embedded in one or more relevantportions of the input text.
 11. The computer-implemented method of claim10, wherein providing the answer further comprises discarding at leastone non-selected portion of the question upon determining that thenon-selected portion of the question is at least one of irrelevant toproviding the answer to the question, or redundant to at least a part ofthe selected relevant portions of the question.
 12. Acomputer-implemented method of performing text-to-text transformation,the method comprising: receiving training examples related to anoriginal text of a specific task; performing a text-to-texttransformation operation of an original text to generate trainingexamples of a transformed text; training a machine learning model toperform Natural Language Processing (NLP) based upon a training datacomprising the training examples of the original text and the trainingexamples of the transformed text; receiving a test data comprising a neworiginal text for transformation; and transforming the new original textinto a new transformed text using the trained machine learning model.13. The computer-implemented method of claim 12, wherein the generatedtraining examples of the transformed text comprises summarizing theoriginal text.
 14. The computer-implemented method of claim 12, whereinthe generated training examples of the transformed text comprisessummarizing and shortening the original text.
 15. Thecomputer-implemented method of claim 12, wherein the specific taskcomprises at least one target task provided in the original text; andthe method further comprising: performing a distant supervision of thetext transformation operation based on a set of related auxiliary taskshaving labeled data in addition to a data of the at least one targettask.
 16. A computing device configured to perform text-to-texttransformation, the computing device comprising: a processor; a memorycoupled to the processor, the memory storing instructions to cause theprocessor to perform acts comprising: performing a text transformationoperation on an original input text of a specific task to generate aplurality of transformed text; applying a task-specific performancemetric to measure an operation of the specific task using each one ofthe plurality of transformed text; pairing each of the plurality oftransformed text with the task-specific performance metric; and updatinga training dataset to include each pairing of the plurality oftransformed text with the task-specific metric.
 17. The computing deviceof claim 16, wherein: the specific task comprises at least one targettask provided in the original text; and the instructions cause theprocessor to perform an additional act, comprising performing a distantsupervision of the text transformation operation based on a set ofrelated auxiliary tasks having labeled data in addition to a data of theat least one target task.
 18. The computing device of claim 16, wherein:the labeled data of the related auxiliary tasks includes question andanswer (QA) pairs; and the instructions cause the processor to performadditional acts, comprising: generating a plurality of questioncompression candidates from the QA pairs; and selecting one or more ofthe compression candidates based on an answer ranking of the QA pairs.19. The computing device of claim 18, wherein: the question compressioncandidates are based on a number of words in the original text; and theinstructions cause the processor to perform an additional act,comprising compressing questions in the text transformation operation byproviding a summary of the question compression candidates using fewerwords than in the original text.
 20. The computing device of claim 16,wherein the instructions cause the processor to perform additional acts,comprising: performing a new text transformation operation using theupdated training dataset to generate a new plurality of transformedtext; applying the task-specific performance metric to measure anoperation of the specific task using each one of the new plurality oftransformed text; pairing each of the new plurality of transformed textwith the task-specific performance metric; and updating the trainingdataset to include each pairing of the new plurality of transformed textwith the task-specific metric.